Serial No. 10/671,081 



Janani Janakiraman 



Page 9 of 22 



Section III: 

AMENDMENT UNDER 37 CFR §1.121 to the 
DRAWINGS 



No amendments or changes to the Drawings are proposed. 
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Section IV: 
AMENDMENT UNDER 37 CFR §1.121 
REMARKS 

Objections to the Specification 

In the Office Action, an objection to the specification for failing to provide antecedent 
basis for the claim term "computer readable medium", repeated from the first Office Action. In 
reply to the first instance of this objection, Applicant had argued: 

Applicant's Previous Reply: 

The term "computer readable medium" and its plural version "computer readable media" 
are well-known term in the art. These tenns are recognized throughout USPTO literature and 
guidelines, such as at MPEP 2106.01, where it is specifically stated (emphasis added): 

"... When functional descriptive material is recorded on some 
computer-readable medium , it becomes structurally and functionally 
interrelated to the medium and will be statutory in most cases since use of 
technology permits the function of the descriptive material to be realized. 
Compare In re Lowry, 32 F.3d 1579, 1583-84, 32 USPQ2d 1031, 1035 (Fed. 
Cir. 1994)(discussing patentable weight of data structure limitations in the 
context of a statutory claim to a data structure stored on a computer readable 
medium that increases computer efficiency) and Warmerdam, 33 F.3d at 
1360-61, 31 USPQ2d at 1759 (claim to computer having a specific data structure 
stored in memory held statutory product-by-process claim) with Warmerdam, 33 
F.3datl361,31 USPQ2d at 1760 (claim to a data structure per se held 
nonstatutory)." 

Applicants have similarly disclosed example embodiments comprising software stored 
on, in, or encoded by computing platforms (paragraph [0034]) which include memory and/or disks 
(paragraphs [0037], [0038]). Such software can also be received by a computing platform via an 
interface, such as a LAN or wireless interface (paragraphs [0039] - [0040]). 

Whereas the term "computer readable medium" is well-known in the art, well-recognized 
by the USPTO, and whereas sufficient examples of embodiments comprising computer readable 
media are specifically recited in applicants' disclosure, there is sufficient antecedent basis to 
support the term's use in claims. Withdrawal of the objection is respectfully requested. 
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In the present Office Action, the Examiner has responded to these points by quoting 
MPEP 608.01(o), but has not provided further explanation as to why a term which is used within 
the USPTO's own literature and policies must be defined in a patent disclosure in order to recite 
it in a claim, and has not suggested alternative language which the Examiner would consider 
acceptable. 

By "computer-readable medium", Applicant is referring to the commonly used range of 
things which can be encoded with software such that a computer can "read" or retrieve the 
software for execution. For example, the publicly-edited resource Wikipedia.com demonstrates 
wide understanding of these terms (emphasis added by Applicant): 

Encyclopedia: 

Machine-readable (Redirected from Computer readable) The term 
machine-readable (or computer-readable) refers to information 
encoded in a form which can be read (i.e., scanned/sensed) by a 
machine/computer and interpreted by the machine's hardware and/or 
software. Machine-readable technologies include optical character 
recognition (OCR) and barcodes. 

Machine-readable medium In telecommunication, a machine-readable 
medium (automated data medium) is a medium capable of storing data 
in a form that can be accessed by an automated sensing device. 

Examples of machine-readable media include (a) magnetic disks, cards, 
tapes, and drums, (b) punched cards and paper tapes, (c) optical disks, 
(d) barcodes and (e) magnetic ink characters. 
Also see: Machine-readable 

References: This article contains material from the Federal Standard 
1037C, which, as a work of the United States Government, is in the 



public domain. 
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And, following Wikipedia's reference to Federal Standard 1037C, one can see that terms 
have been widely used and documented since at least 1996: 

Federal Standard: 

machine-readable medium: A medium capable of storing data in a form 
that can be accessed by an automated sensing device. Note: Examples 
of machine-readable media include (a) magnetic disks, cards, tapes, and 
drums, (b) punched cards and paper tapes, (c) optical disks, and (d) 
magnetic ink characters. Synonym automated data medium . 

medium: 1. In telecommunications, the transmission path along which a 
signal propagates, such as a wire pair, coaxial cable, waveguide, optical 
fiber, or radio path. (188) 2. The material on which data are or may be 
recorded, such as plain paper, paper tapes, punched cards, magnetic 
tapes, magnetic disks, or optical disks. 
(Source: Federal Standard 1037, accessed from 
http://www.its.bldrdoc.gov/fs-1037/which notes this definition as last 
generated on Fri Aug 23 00:22:38 MDT 1996) 

The Institute for Telecommunication Sciences (ITS) is the research and engineering 
branch of the National Telecommunications and Information Administrations (NTIA), which is 
part of the U.S. Department of Commerce, and thus is a cousin organization to the USPTO. 

Applicant has disclosed such examples of computer-readable or machine-readable media 
(see paras. 0037 - 0040) in the form of memory, disks, and various network interfaces. To 
advance prosecution, Applicant hereby amends the disclosure to specifically recite "computer- 
readable media" in these paragraphs. Applicant believes no new matter is entered by these 
changes. 

However, should the Examiner disagree, Applicant respectfully requests the Examiner to 
suggest alternative claim language which would be agreeable to the Examiner. 

MPEP 2173.02: . . . When the examiner is satisfied that patentable 
subject matter is disclosed, and it is apparent to the examiner that the 
claims are directed to such patentable subject matter, he or she should 
allow claims which define the patentable subject matter with a reasonable 
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degree of particularity and distinctness. Some latitude in the manner of 
expression and the aptness of terms should be permitted even though 
the claim language is not as precise as the examiner might desire. 
Examiners are encouraged to suggest claim language to applicants to 
improve the clarity or precision of the language used, but should not 
reject claims or insist on their own preferences if other modes of 
expression selected by applicants satisfy the statutory requirement. 

Applicant respectfully requests withdrawal of the objection to the specification. 

Rejections under 35 U.S.C. S103 

In the Office Action, rejections were maintained of claims 1 - 16 under 35 U.S.C. § 103(a) 
as being unpatentable over US Published Patent Application 2004/0143430 to Said, et al. 
(hereinafter "Said"), in view of US Patent 6,876,728 to Kredo, et al. (hereinafter "Kredo"). 

Claim Amendments Are Patently Distinct over Said in view of Kredo. To advance 
examination to allowance, Applicant is voluntarily amending the independent claims to further 
recite the invention's aspects of being able to automatically insert different emoticons for a 
sender's emotions based on a profile of each recipient in a situation where a single message is 
sent from an author to multiple recipients. In such a scenario, a profile for each recipient is used 
to insert an appropriate icon for each recipient. AIM's emoticon tool bar can only insert the 
same emoticon in all copies of a message directed to multiple recipients because it does not 
consider a recipient profile. Said-provisional and Kredo fails to suggest or teach such multiple- 
recipient profile use and recipient-specific emoticon insertion, as well. 

Applicant requests allowance of all claims as amended without prejudice, and reserves 
the option to file the current claim language in a related patent application. 

Tool Bar for Manually Inserting Emoticons Not Anticipatory of Claim Elements and 
Steps for Automatic Insertion of Emoticons. In reply to the rejections over Said in view of 
Kredo in the first Office Action, Applicant challenged whether or not Said's provisional 
application (hereinafter "Said-provisional") supported the portions of Said's published non- 
provisional application (hereinafer "Said-non-provisional"). Otherwise, Said-non-provisional 
would not qualify as prior art to Applicant's patent applicadtion. 

The Examiner responded in the present Office Action by referring generally to a figure in 
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the Said-provisional which shows an image of an instant messenger interface having a braille 
translation and a tool bar for inserting emoticons. 

The referenced figure shown on page 10 of Said-provisional only shows a well known 
user interface for the America Online Instant Messenger ("AIM") user interface, which includes 
a user-operable tool bar through which an authoring user can manually select an emoticon to 
insert in a message. These emoticons, however, are not automatically inserted responsive to 
"receiving one or more emotional characteristic indicators from at least one emotional content 
analyzer system as the result of at least one emotional content analysis performed on said 
communication session from said first person" as claimed, rather they are inserted manually by 
the message author. Said-provisional is merely showing on page 10 a "mock-up" of a user 
interface. The term "mock-up" is specifically stated in Said-provisional at pg. 9, last paragraph. 

And, the user interface diagram also fails to indicate whether Said had possession of the 
ability in implement the braille translator as prophesied by their "development plan" (please refer 
to the following paragraphs regarding disqualification of Said's "proposal" as a prophetic, non- 
enabling disclosure). It is not clear if the braille dots in fact represent any form of emotional 
content or perhaps are just a translation of the phrase "Hello friend, how are you?". 

Because Said-provisional fails to provide filing date support for this claim step or 
element, Said-nonprovisional is disqualified as prior art. Applicant respectfully requests 
withdrawal of all rejections over Said. 

"Expert Analysis" is Required to Arrive at Said's Predicted Results. In the present Office 
Action, the Examiner has argued that: 

Examiner in the Office Action: 

. . . The general user can see from the picture [pg. 10 of Said-provisional] 
in the provisional application, without an in-depth expert analysis, 
showing the support for the subject matter in the Said reference used in 
the rejection. . . . 

Applicant respectfully disagrees. Applicant submits that the Examiner's conclusion of 
what would not require "expert analysis" is contrary to the teachings of Said-provisional because 
(1) Said-provisional is not actually stating that such emoticon insertion technology has been 
completed, and (2) Said describes such implementation of emotional content analysis as 
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"complex" (emphasis added by Applicant): 
Said-provisional: 

. . . First, the language is not just spoken with hand movements, but 
includes movements and positioning of almost all parts of the body. 
Second, facial expressions play a major role in certain of the signed 
words and concepts. Third, nonverbal communication such as body 
language is used in ASL as well to convey the same types of things (such 
as emotional content) that hearing people use body language for as well. 
Finally, an entirely separate problem from collecting the data about the 
person signing exists in the actual processing and translation of that data. 
The grammar and syntax of ASL is as different from English as Japanese 
or German is from English, meaning that the order and relationship of 
words that come out of the translation may be easily jumbled or 
juxtaposed and hence not make much sense. As with foreign language 
translation, subtle nuances, context-sensitive words, and figures of 
speech may also not translate properly. Designing a system to capture 
all of these features will be quite complex. 

Several existing research projects are tackling this from a variety 
of angles. The majority of existing research projects can be divided into 
two categories. . . . 

Whereas the Examiner has not established what the ordinary level of skill in the art at the 
time of the present invention, whereas no extrinsic evidence has been entered to suggest that 
designing a emotional content analyzer to automatically insert emoticons into a message based 
on analysis of content of the message would be trivial or a matter of ordinary design practice, 
and whereas the extrinsic evidence of record (Said-provisional) clearly indicates such "will not" 
be the result of non-"expert analysis", Applicant respectfully suggests that it would have been an 
unreasonable leap in reasoning for an ordinarily skilled person to make the necessary advances 
provided a simple "mock-up" of an AIM user interface such as that in Said's provisional patent 
application. For this reason, Applicant requests allowance of all claims over Said. 

Support For Relied-upon Portions of Said Reference Not Found in Provisional 
Application. In the present Office Action, Said's non-provisional patent application disclosure at 
paragraphs [0003], [0012], [0014], [0029], and [0033] were relied upon for the rationale of the 
rejections. 
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In general, the "proposal for development" filed by Said, et al, as a provisional patent 
application does not resemble the format of a patent application. Said-non-provisional contains 
only one figure, a flow chart, which is not shown in Said-provisional. 

Specifically, paragraph [0003] sets forth a field of the invention, which is supported by 
Said-provisional's forward-looking and prophetic statements regarding the goals of the 
"proposal". 

Paragraph 0012 appears in Said-provisional, but only discloses historical uses of instant 
messaging and limitations of it being similar to an "old-fashioned" telegraph. 

Paragraph 0014 refers to users/authors using instant messaging to manually enter "raw 
text information", and user-input (not automatic) of non-verbal meaning (e.g. emotional 
indicators). Said-provisional only supports manual insertion of emoticons as shown in the AIM 
"mock-up" user interface on page 10 of Said-provisional. Paragraph 0014 does not appear in 
Said-provisional, and thus is not supported as prior art against Applicants claims. While is it 
common for a user/author to be able to manually insert an emoticon into a message, it is 
previously unknown in the art for a system to automatically analyze the emotional content of a 
message to determine an author's emotional state, and to insert an appropriate emoticon 
automatically (not manually) to appropriately communicate this to a plurality of recipients based 
on profiles of the author and of each recipient. 

Paragraph 0029 refers to "common" use of emoticons in XML-based text displays of 
messaging systems. Said-provisional only supports such "common" use (manual insertion of 
emoticons), as shown in the AIM "mock-up" user interface on page 10 of Said-provisional. 
Paragraph 0029 does not appear in Said-provisional, and thus is not supported as prior art against 
Applicants claims. While is it common for a user/author to be able to manually insert an 
emoticon into a message, it is previously unknown in the art for a system to automatically 
analyze the emotional content of a message to determine an author's emotional state, and to 
insert an appropriate emoticon automatically (not manually) to appropriately communicate this 
to a plurality of recipients based on profiles of the author and of each recipient. 

Said's paragraph [0033] refers to Virtual Sign Language and American Sign Language, 
and refers to providing a facial expression on an avatar (not an emoticon as claimed) to represent 
the signing author's emotional state. Further, it is not clear from paragraph 0033 whether Said 
is referring to traditional manual encoding of a signer's facial expression or is referring to an 
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automated facial expression recognition process. Most importantly, paragraph [0033] does not 
appear in Said-provisional. 

For these reasons, Applicant respectfully submits that Said-provisional fails to support all 
of the relied upon portions of Said-non-provisional for the rejections, and that Said fails to teach 
or suggest all of the claim elements, steps, and limitations. For these reasons, allowance of all 
claims is requested. 

Said Non-Provisional Patent Application Reference Is Not Qualified Prior Art because 
Said-Provisional is Prophetic and is Non-Enabling. In the rationale for the rejections over Said 
in view of Kredo in the first Office Action, Said's pre-grant published patent application number 
2004/0143430 (PG-Pub) was relied upon in the proposed combination under 35 U.S.C. § 103(a). 
However, the filing date of Said's non-provisional patent application is October 15, 2003, which 
falls after applicants' filing date of September 25, 2003. 

Although it was not explicitly stated in the first Office Action, presumably the examiner 
was affording the relied-upon portions of Said's PG-Pub the filing date of Said's provisional 
patent application (PPA), which was October 15, 2002. 

Applicant suggested that the relied-upon portions of Said's published patent application 
may not be due priority to Applicant's filing date because it was not established in the rationale 
for the rejections where support for the relied-upon portions of Said's published patent 
application is believed to be found by the Examiner in Said's provisional patent application. 
Thus, since the burden of establishing a prima facie case of obviousness falls on the Examiner, 
Applicant requested the Examiner to provide details of where it is believed such support is found 
in Said's provisional patent application. 

In the present Office Action, the Examiner has repeated the technical rationale for the 
rejections, and has noted that the Applicant has chosen not to argue technical differences but to 
focus on availability of the Said published patent application as prior art. Applicant agrees with 
the Examiner's statement. Applicant notes that there is no requirement in the law to respond to 
technical details and comparisons of references which have not been established as being prior 
art. Applicant is aware of the Patent Office's policy to afford blanket filing date priority of all 
disclosure details in non-provisional patent applications to filing dates of cross-referenced 
provisional patent applications. 

However, Applicant also notes that during a one-year pendency of a provisional patent 
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application, most system designs evolve and are refined, so that a later- filed non-provisional 
patent application often reflects a more recent, and often more complete design. As such, many 
non-provisional applications are not fully supported for all they teach by their counterpart 
provisional applications. Applicant also notes that provisional patent applications are not 
examined for verified for their compliance with §112, both for content (e.g. enablement, best 
mode, etc.), or for format. By reviewing many provisional patent applications, one can see a 
wide range of submissions which are not in actual patent application format, but often represent 
other types of business documents (proposals, presentations, white papers, etc.). 

Applicant notes that use of future tense verbs in a disclosure is customarily interpreted to 
indicate prophetic results (e.g. results which are expected, but which have not been achieved or 
verified), while present tense statements are not indicative of either prophetic or actual results, 
and post tense statements are considered to be affirmative statements of actual results and actual 
accomplishments. This is a standard for interpretation of patent disclosures recognized in US 
patent examination procedure, and in numerous Court decisions. MPEP 608.0 l(p)II. 
Hoffman-La Roche, Inc. v. Promega Corp. (Fed. Cir. 2003). Pharmacia Ent. Corp. v. Par 
Pharmaceutical (Fed. Cir. 2005). Novo Nordisk Pharmaceuticals, Inc. v. Bio-Technology 
General Corp. (Fed. Cir. 2005). 

Applicant, therefore, respectfully submits that Said's provisional patent application is 
merely a document containing prophetic results, and that it relies heavily upon future technology 
which was as-of-yet undeveloped, and thus it does not qualify to support Said's non-provisional 
patent application for the purpose of affording it filing priority over Applicant's patent 
application (e.g. it does not rise to a level of support to establish Said's non-provisional teachings 
as prior art). 

First, Applicant respectfully requests that the Examiner consider the nature and format of 
Said's provisional patent application. Clearly, it is not in the format of a patent application in a 
traditional manner. But, even more convincing is its title, which does not state "patent 
application", but instead states (emphasis added by Applicant): 

Said Provisional Application: 

"A Product Development and Commercialization Proposal " 



This is not the only signal that possibly this is a prophetic document - the word "plan" is 
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used throughout the document, as are future tense descriptions of planned design activities, 
technical features, etc. (e.g. pg. 3 "will be the development", "could include", "would be", "could 
be developed", "primary plan"; pg. 4 "would consist of the development of the core software 
technologies", "would be developed"; pg. 10 "would be", "would allow", could function", 
"would rely on"; etc.). 

As such, Applicant respectfully submits that this document is a proposal and a plan for a 
project which was filed as a provisional patent application for the purposes of establishing 
evidence of dates of conception but not actual reduction to practice. While a patent application's 
filing date is often treated as a date of constructive reduction to practice, this presumption may 
not be afforded to provisional patent applications because provisional patent applications are not 
examined and are often not in compliance with §112 with respect to enablement, best mode, etc. 

Applicant also notes that a patent application cannot comply with the written description 
requirement for portions of technology which was developed subsequent to the filing of the 
patent application because an inventor could not have possessed the invention at the time of 
filing. Chiron Corp. V. Genentech, Inc., 363 F.3d 1247, 70 USPQ2d 1321 (Fed. Cir. 2004), 
petition for cert, filed, October 4, 2004. 

Said's provisional patent application disclosure relies upon, and heavily refers to future 
developments in the areas of sign language recognition and emotion analysis, and does not 
purport to disclose these actual inventions because the inventors recognize their incomplete state 
and plan to rely on (e.g. integrate) technologies from others working in these fields (emphasis 
added by Applicant): 

Said Provisional Application: 

The final stage involves Sign Language Recognition, a 
technology still in the research stages that allow a computer to 
recognize a human signer and convert that information into text." (Pg. 4) 

Many existing research ideas are currently being pursued to 

accomplish the same end: allowing a human signer to automatically 
convert the gestures, motions, and expressions of sign language into an 
electronic text format. (Pg. 10) 



. Third, nonverbal communication such as body language is used in 
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ASL as well to convey the same types of things (such as emotional 
content) that hearing people use body language for as well. Finally, an 
entirely separate problem from collecting the data about the person 
signing exists in the actual processing and translation of that data. The 
grammar and syntax of ASL is as different from English as Japanese or 
German is from English, meaning that the order and relationship of words 
that come out of the translation may be easily jumbled or juxtaposed and 
hence not make much sense. As with foreign language translation, subtle 
nuances, context-sensitive words, and figures of speech may also not 
translate properly. Designing a system to capture all of these features 
will be guite complex. 

. . . Several existing research projects are tackling this from a variety 
of angles . The majority of existing research projects can be divided into 
two categories. . . . (Pg. 11) 

GSR - This is the one area that we will need to do some research and 
development on . Hence it will not be part of the first phase of the 
product. (Pg. 21) 

Clearly, from all of the references here, dependency upon projects which are still in 
research by others, at least some, if not all of Said's provisional disclosure is signaling that the 
inventors are not in possession of this technology yet, but plan to be ready for it when they can 
acquire it from others. 

For these reasons, Applicant respectfully submits that Said's provisional patent 
application should not be presumed to provide evidence as constructive reduction to practice 
because it only predicts intended project results, and it sets forth intended steps to investigate 
and develop actual designs, but it does not represent itself as having accomplished or completed 
the design goals and objectives it outlines. Thus, Applicant submits that each relied-upon 
portion of Said's non-provisional patent application must be considered closely to determine if 
full, §1 12-compliant support is present in Said's provisional patent application, otherwise, Said's 
non-provisional patent application is not qualified as prior art against Applicant's patent 
application. 

Teachings of Said and Kredo not Available to Ordinarily Skilled Artisans prior to 
Applicant's Filing Date. In response to these rejections in the first Office Action, Applicant 
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argued that both Said an Kredo would not have been available to one of ordinary skill in the art, 
whereas the dates of filing and publication indicate that only the inventors of the Said reference 
would have had access to the information in both references. And, Applicant established, using 
extrinsic evidence, that the inventors of the Said reference were not "ordinarily skilled". 

The Examiner disagreed in the present Office Action. While Applicant maintains this 
argument is valid and should be dispositive of non-obviousness of the claimed invention, 
Applicant believes that through the present RCE the Examiner will find agreement with the 
present amendment and following remarks. 

Ordinary Level of Skill in the Art Not Established by the Examiner. In reply to the first 
Office Action, Applicant noted that the Examiner had not established what level of skill in the 
art was deemed to be "ordinary" at the time of filing the present patent application. In the 
second Office Action, the Examiner has not responded and has not completed this portion of the 
required analysis under Graham v. John Deere Co. 

With respect to the determination of obviousness under 35 U.S.C. 103(a), the Examiner is 
a fact finder required to resolve Graham inquiries ("Examination Guidelines for Determining 
Obviousness Under 35 U.S.C. 103 in View of the Supreme Court Decision in KSR International 
Co. v. Teleflex Inc.:, Fed. Reg., Vol. 72, No. 195, October 10, 2007). 

According to the Court in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), 
it is critical to determining obviousness under 35 U.S.C. §103 to ascertain the level of ordinary 
skill in the art, whereas this is pivotal in the language and standard set forth in the law at §103. 

In the Office Action, the Applicant has not been notified what was considered to be the 
level of ordinary skill in the art. It is not clear if any of the criteria to be considered in 
determining the level of ordinary skill in the art under the third factual inquiry of Graham v. 
John Deere, as set forth in Environmental Designs, Ltd. v. Union Oil, 713 F.2d 693, 696, 218 
USPQ 865, 868 (Fed. Cir. 1983) and in Bausch & Lomb, Inc. v. Barnes-Hind/Hydrocurve, Inc., 
796 F.2d 443, 449-450, 230 USPQ 416, 420 (Fed. Cir. 1986) such as (1) the educational level of 
the inventor, (2) the type of problems encountered in the art, (3) the prior art solutions to those 
problems, (4) the rapidity with which innovations are made by others, not including the inventor, 
(5) the sophistication of the technology, and (6) the educational level of active workers in the 
field not including the inventor, were considered, and if so, how. 
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Applicant respectfully submits that a holding of obviousness is improper without this 
factual determination by the Examiner, and requests withdrawal of all rejections until Applicant 
has been informed of the results of such analysis. 

Applicant respectfully requests continued examination of this patent application, entry of 
the amendment, a allowance of all pending claims. 



Respectfully submitted, 




Robert H. Frantz, Reg. No. 42,553 
Agent for Applicant 
Tel: (405) 812-5613 
Franklin Gray Patents, LLC 



